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Modern  military  operations  require  interoperability.  The  Department  of  Defense 
(DoD)  has  made  tremendous  interoperability  gains  over  the  last  few  years. 
Unfortunately,  without  a  way  to  assess  the  status  of  interoperability  throughout 
the  department,  it  is  difficult  to  quantify  this  progress.  Although  interoperability 
issues  are  persistent  and  visible,  the  number  of  interoperability  successes  is 
easily  overlooked.  Most  systems  developed  today  meet  the  interoperability 
requirements  that  were  specified  in  their  operational  requirements  documents 
(ORDs).The  application  of  a  set  of  metrics  addressing  this  domain  would  shed 
more  light  on  the  situation  and  highlight  the  successes  of  the  many  agencies 
that  have  labored  to  produce  interoperable  systems.  Effective  metrics  would 
enable  the  services  and  agencies  to  make  informed  decisions  about  the 
allocation  of  scarce  resources  to  solve  interoperability  in  already  fielded  systems. 


The  pejorative  use  of  the  term  “legacy 
system”  often  occurs  when  commu¬ 
nications  and  computer  systems  are 
described.  This  is  unfortunate  because 
many  fielded  systems  are  performing  well 
and  meeting  or  exceeding  their  original 
specifications. 

“C4ISR”  refers  to  systems  that  are  part 
of  the  Command,  Control,  Communica¬ 
tions,  Computer,  Intelligence,  Surveil¬ 
lance,  and  Reconnaissance  domain.  The 
C4ISR  domain  is  one  of  four  domains 
for  which  the  Joint  Technical  Architecture 
specifies  a  domain  annex.  C4ISR  is 


defined  in  the  Joint  Technical  Architecture 
(JTA;  Defense  Information  Systems 
Agency,  1999)  as  those  systems  that 

•  support  properly  designated  command¬ 
ers  in  the  exercise  of  authority  and 
direction  over  assigned  and  attached 
forces  across  the  range  of  military 
operations; 

•  collect,  process,  integrate,  analyze, 
evaluate,  or  interpret  available  informa¬ 
tion  concerning  foreign  countries  or 
areas; 
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Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 
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•  systematically  observe  aerospace,  sur¬ 
face  or  subsurface  areas,  places,  per¬ 
sons,  or  things  by  visual,  aural,  elec¬ 
tronic,  photographic,  or  other  means; 
and 

•  obtain,  by  visual  observation  or  other 
detection  methods,  information  about 
the  activities  and  resources  of  an  en¬ 
emy  or  potential  enemy,  or  secure  data 
concerning  the  meteorological,  hydro- 
graphic,  or  geographic  characteristics 
of  a  particular  area. 

The  JTA  specifies  a  minimal  subset  of 
interoperability  requirements.  It  is  becom¬ 
ing  trite  to  say  that  the  JTA  is  necessary 
but  not  sufficient  to  achieve  interoper¬ 
ability.  Interoperability  is  a  hot-button 
issue  and  it  certainly  should  be.  However, 
as  Col.  Thomas  Andrew,  U.S.  Air  Force, 
has  observed,  “There  is  a  lot  of  inter¬ 
operability  out 
there.  Many 
C4ISR  systems 
do  interoperate 
quite  well  to¬ 
gether.”  As  the 
Commander  of 
Defense  Infor¬ 
mation  Sys¬ 
tems  Agency’s 
(DISA’s)  Joint 
Interoperability  Test  Command  (JITC), 
Col.  Andrew  is  in  a  strong  position  to 
speak  with  authority,  since  he  has  the 
responsibility  as  the  DoD’s  sole  certifier 
of  joint  interoperability  for  systems. 

The  continued  accelerated  advancement 
of  information  technology  ensures  that 
fielded  systems  do  not  have  the  latest  and 
greatest  capabilities.  The  revolution  in 
military  affairs  is  rapidly  accelerating  the 


rate  at  which  requirements  change,  but  the 
essential  question  should  be  does  the  fielded 
system  meet  mission  requirements? 

The  revolution  in  military  affairs  is  built 
on  software.  The  rapid  linking  of  dispar¬ 
ate  weapons  and  command  systems  is 
done  via  software.  Therefore,  a  significant 
number  of  interoperability  issues  are  soft- 
ware-based.  Laymen  commonly  think  of 
software  in  terms  of  application  software. 
More  often  than  not,  interoperability  is¬ 
sues  dealing  with  passing  targeting  data 
from  a  sensor  platform  to  a  weapons  plat¬ 
form  (sensor-to-shooter)  involve  low-level 
software  to  include  firmware.  Firmware 
is  essentially  software-reprogrammable 
chipsets. 

Rapid  technological  advances  have  also 
fueled  the  revolution  in  business  affairs. 
Innovative  solutions  are  sought  to  accel¬ 
erate  the  fielding  of  new  technology.  Com¬ 
mercial  off-the-shelf  (COTS)  software  is 
widely  touted  as  the  silver  bullet  for  speed¬ 
ing  the  delivery  of  updated  software  to  the 
field.  Unfortunately,  there  are  no  COTS 
products  for  purely  military  applications 
such  as  embedded  weapons  systems.  Even 
with  application  software,  some  commer¬ 
cial  products  produce  interoperability 
problems  because  they  are  designed  to  be 
proprietary  closed  systems. 

Defining  the  Problem  Space 


Given  the  enormous  number  of  C4ISR 
systems  in  use  in  today’s  armed  forces,  it 
is  critical  that  we  understand  clearly  which 
systems  are  being  addressed  by  our  ap¬ 
proach.  This  section  establishes  the  scope 
of  the  problem  with  which  this  article  will 
concern  itself. 


"The  continued 
accelerated 
advancement  of 
information  tech¬ 
nology  ensures  that 
fielded  systems  do 
not  have  the  latest 
and  greatest 
capabilities." 
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When  systems  are  fielded  from  outside 
the  DoD  acquisition  process,  interoper¬ 
ability  responsibility  for  these  systems  is 
also  outside  the  DoD  acquisition  com¬ 
mands.  In  this  paper  we  discuss  C4ISR 
interoperability  for  systems  that  have  been 
fielded  through  project  managers  (PMs) 
and  program  executive  officers  (PEOs). 

Our  general  approach  is  to  narrow  the 
field  to  C4ISR  systems;  ensure  that  these 
systems  have  interoperability  require¬ 
ments;  and  ensure  that  we  focus  on  com¬ 
bat  requirements.  We  discuss  each  of  these 
elements  in  greater  detail,  and  suggest  a 
methodology  for  arriving  at  the  “right”  list 
of  systems. 

What  is  a  C4ISR  system?  Our  approach 
is  concerned  with  C4ISR  systems  and  may 
not  be  applicable  to  other  types  of 
interoperability.  The  JTA’s  definition  of 
C4ISR  systems  is  presented  in  the  intro¬ 
duction.  C4ISR  systems  move  data  that  is 
critical  to  the  conduct  of  military  opera¬ 
tions.  Information  systems  that  do  not 
move  this  type  of  data  have  no  operational 
requirement  to  be  interoperable.  (There 
may  be  a  functional  requirement  for  rea¬ 
sons  of  organizational  efficiency,  but  this 
is  distinctly  different  from  an  operational 
requirement.) 

The  task  of  identifying  all  C4ISR  sys¬ 
tems  seems  daunting.  C4ISR  systems  are 
developed  by  a  dizzying  array  of  organi¬ 
zations  within  the  DoD.  Fortunately,  the 
preparations  for  the  Year  2000  crisis 
resulted  in  extensive  efforts  throughout  the 
DoD  to  inventory  all  computer  and  commu¬ 
nications  systems  throughout  the  depart¬ 
ment.  These  databases  can  be  leveraged 
to  provide  an  initial,  all-inclusive  list  of 
systems.  Applying  the  definition  of  C4ISR 
systems  provided  in  the  JTA  can  narrow  this 
list  of  computer  and  information  systems 


to  those  systems  that  perform  C4ISR 
functions. 

The  second  step  is  to  eliminate  prob¬ 
lems  that  are  not  interoperability  problems. 
Interoperability  is  defined  by  the  IEEE  as 
“the  ability  of  two  or  more  systems  or 
components  to  exchange  data  and  use  in¬ 
formation”  (IEEE,  1990).  This  road  map 
does  not  apply  to  deficient  capability  that 
is  isolated  to  a 
particular  sys¬ 
tem.  Rather,  it 
focuses  on  situ¬ 
ations  in  which 
the  ability  for 
multiple  sys¬ 
tems  to  communicate,  cooperate,  or  coex¬ 
ist  is  lacking.  At  the  same  time,  it  is  im¬ 
portant  that  we  do  not  construe  the  poten¬ 
tial  for  interoperability  too  narrowly  — 
most  systems  have  interaction  with  other 
systems  at  some  level.  A  system  should 
be  eliminated  at  this  point  only  if  it  meets 
none  of  the  following  criteria: 

•  It  generates  data  that  is  used  by  another 
system. 

•  It  processes  or  consumes  data  that  is 
generated  by  another  system. 

•  It  relies  on  another  system  for  delivery 
of  data. 

•  It  is  software  that  operates  on  the  same 
platform  as  another  system. 

For  legacy  systems,  these  criteria  may 
narrow  the  list  of  systems  considerably. 
However,  in  today’s  changing  environ¬ 
ment,  this  filter  may  quickly  become  less 
effective.  Because  of  the  movement  to 
network-centric  systems  required  by  Joint 


"The  task  of 
identifying  all 
C4ISR  systems 
seems  daunting." 
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Visions  2010  and  2020  (JV2010/JV2020), 
there  will  be  increasing  interaction  be¬ 
tween  systems  that  were  previously  dis¬ 
joint.  In  addition,  the  proliferation  of  ap¬ 
plications  and  the  need  for  lighter,  more 
agile  components  will  drive  more  and 
more  applications  away  from  dedicated 
computing  platforms  and  onto  common 
ones.  This  will  drive  the  need  for  shared 
service  between  applications  that  were 
previously  disjoint. 

Finally,  we  wish  to  focus  on  systems 
that  are  critical  to  the  unified  commands. 
Problems  that  do  not  affect  the  sharp  end 
of  the  spear  clearly  deserve  less  attention. 
We  are  interested  in  an  approach  to 
interoperability  that  increases  the  combat 
capability  of  the  U.S.  Armed  Forces.  Sys¬ 
tems  that  directly  impact  our  ability  to 
accomplish  our  mission  should  be  the  first 
on  our  list.  We  can  accomplish  this  by 
considering  the  joint  mission  essential  task 
lists  (JMETLs)  of  the  unified  commands. 
If  a  particular  system  does  not  help  to 
accomplish  a  JMETL  for  a  unified  com¬ 
mander,  it  can  be  safely  eliminated  from 
consideration. 


Thus,  we  recommend  a  systematic, 
three-step  methodology.  First,  each  com¬ 
mand  responsible  for  C4ISR  procurements 
should  identify  the  universe  of  all  C4ISR 
systems.  Next,  eliminate  those  systems  for 
which  interoperability  is  not  an  issue. 
Finally,  compare  the  list  to  the  JMETL  of 
the  unified  commands,  using  this  to  elimi¬ 
nate  systems  that  are  either  not  joint  or 
not  mission  essential.  A  diagram  describ¬ 
ing  the  process  by  which  we  arrive  at  the 
final  group  of  systems  is  shown  in  Figure 
1. 

Considering  Existing  Metrics 

Interoperability  is  notoriously  difficult 
to  measure.  Although  one  might  at  first 
think  that  interoperability  is  an  all-or-noth- 
ing  proposition,  this  is  an  oversimplifica¬ 
tion.  For  example,  systems  that  can  ex¬ 
change  all  of  the  required  data  elements 
might  be  interoperable  —  but  if  the  speed 
of  the  exchange  is  too  slow  to  support  the 
operational  requirements,  then  the  so- 
called  interoperability  will  not  be  of  oper¬ 
ational  value.  Sometimes  interoperability 


Army  C4ISR 

Systems 

Navy  C4ISR 

JMETL 

Systems 

USAF  C4ISR 

Systems 

Figure  1.  JMETL  validated  C4ISR  Systems 
from  the  Service  PEO/PM  Structure 
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is  realized  through  labor-intensive  work¬ 
arounds.  Are  two  systems  interoperable  in 
this  case?  It  depends  on  a  number  of  fac¬ 
tors:  the  required  frequency;  the  availabil¬ 
ity  of  personnel  to  operate  the  work¬ 
arounds;  and,  in  general,  the  ability  of  the 
procedure  to  meet  the  operational  require¬ 
ments  (both  in  terms  of  effectiveness  and 
suitability). 

There  has  been  at  least  one  attempt  to 
measure  the  interoperability  of  two  sys¬ 
tems,  through  an  effort  called  the  Levels 
of  Information  System  Interoperability 
(LISI).  Rather  than  a  single  measure,  LISI 
is  actually  a  collection  of  related  models, 
a  tool  for  use  in  applying  these  models,  a 
set  of  metrics  and  techniques  for  applying 
the  models,  and  an  initiative  (or  process) 
aimed  at  using  these  models  to  address  a 
wide  set  of  interoperability  objectives.  At 
its  core,  LISI  is  based  around  classifying 
levels  of  interoperability  by  the  “richness” 
of  the  communication  that  a  particular 
system  or  group  of  systems  allows  (C4ISR 
Architecture  Working  Group,  1998).  Al¬ 
though  a  full  discussion  of  LISI  is  beyond 
the  scope  of  this  paper,  we  believe  that  the 
model  is,  at  root,  too  complicated  for  use 
in  aggregating  the  status  of  systems  at  this 
level. 


A  Basis  for  Measurement 


We  propose  a  simplified  model, 
wherein  each  system  will  be  labeled  with 
a  color  code  based  on  two  factors.  The  first 
factor  is  whether  the  system  has  any 
known  interoperability  problems.  A  prob¬ 
lem  exists  if  and  only  if  some  operational 
requirement  cannot  be  met  because  of  the 
deficiency.  Some  capabilities  might  be 
“desirable,”  but  if  they  are  not  required  by 
the  unified  commands,  then  there  is  no 
interoperability  problem  for  the  purposes 
of  our  measure.  This  first  factor,  then, 
measures  whether  the  system  meets  oper¬ 
ational  requirements.  The  second  factor 
is  whether  the  system  meets  its  interoper¬ 
ability  requirement  set.  By  this  we  mean 
that  the  system  has  implemented  all  docu¬ 
mented  interoperability  functionality  and 
has  received  a  joint  certification  from  the 
JITC.  This  factor,  then,  focuses  on  acqui¬ 
sition  requirements.  We  use  this  to  distin¬ 
guish  between  problems  of  requirements 
definition  and  problems  of  requirements 
implementation. 

Using  these  two  methods,  we  arrive  at 
a  four-colored  system,  as  described  in 
Tables  1  and  2. 


Table  1. 

Stop  Light  Model 


Meets  Acquisition  Requirements? 

Yes 

No 

Meets 

Operational 

Requirements? 

Yes 

Green 

Yellow  1 

No 

Orange 

Red 
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Table  2. 

Stop  Light  Color  Definition  and  Imputations 


Green 

The  system  meets  its  interopera¬ 
bility  requirement  set  and  has  no 
known  interoperability  problems 

Fielded  system  without  known 
issues  that  meets  all  documented 
requirements 

Yellow 

The  system  does  not  meet  its 
interoperability  requirement  set, 
but  has  no  known  interoperability 
problems 

Documented  requirements  do  not 
reflect  operational  use  of  the 
system 

Red 

The  system  does  not  meet  its 
interoperability  requirement  set, 
but  has  no  known  interoperability 
problems 

Improvement,  migration  and/or 
action  plans  needs  to  be  put  in 
place 

Orange 

The  system  meets  its  interoper¬ 
ability  requirement  set,  but  has 
known  interoperability  problems 

Revisit  requirements  and  deter¬ 
mine  if  requirements  are 
adequate 

Note  that  use  of  the  stoplight  model 
involves  drawing  hard  lines  between  meet¬ 
ing  and  not  meeting  requirements.  This 
may  not  be  an  entirely  straightforward 
proposition.  Nevertheless,  the  value  of 
drawing  a  line  in  the  sand  between  “ac¬ 
ceptable”  and  “unacceptable”  cannot  be 
overstated.  Only  by  providing  unambigu¬ 
ous  judgments  on  the  status  of  our  sys¬ 
tems  can  we  move  the  debate  from  one  of 
educated  guessing  (i.e.,  rumor  and  innu¬ 
endo)  to  one  with  a  degree  of  rigor  and 
reproducibility. 

There  is  one  important  point  that  needs 
to  be  made  about  this  grading  system.  The 
operational  requirements  considered 
should  be  those  that  are  in  force  at  a 
particular  time.  If  we  are  grading  a  sys¬ 
tem  today,  it  should  be  graded  with  respect 
to  today’s  operational  requirements  —  if 
we  are  projecting  a  grade  for  one  year  from 
now,  it  should  be  based  on  our  best  infor¬ 
mation  about  next  year’s  operational 
requirements. 


However,  the  acquisition  requirements 
used  should  be  those  that  were  to  have 
been  implemented  by  a  particular  point  in 
time.  For  example,  if  we  have  a  radio  sys¬ 
tem  with  a  scheduled  upgrade,  and  the 
requirements  process  was  geared  for  re¬ 
lease  of  the  upgrade  in  one  year,  we  should 
not  include  the  upgraded  requirements  in 
an  evaluation  of  the  system’s  readiness 
today.  We  should,  however,  include  the 
upgraded  requirements  in  an  evaluation  of 
the  system’s  readiness  next  year  —  and 
we  should  do  so  using  our  best  informa¬ 
tion  about  whether/when  the  system  will 
meet  these  requirements.  This  is  impor¬ 
tant  (and  fair)  because  we  must  recognize 
that  the  acquisition  community  is  con¬ 
strained  by  technical  and  fiscal  realities, 
and  does  not  always  have  the  ability  to 
deliver  improvements  in  time.  Its  sched¬ 
ule  is  dictated  by  the  ever-present  trades 
with  cost  and  performance,  and  may  even 
be  influenced  by  the  timeliness  of  the  iden¬ 
tification  of  the  requirement.  This  part  of 
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the  measurement  is  designed  to  measure 
the  acquisition  community’s  ability  to 
respond  to  the  requirements  process  — 
and  schedule  is  a  key  part  of  that  process. 

Of  course,  in  both  cases,  the  farther  in 
the  future  we  project  these  requirements, 
the  more  cautiously  we  must  consider  their 
results.  Nevertheless,  short-  and  medium- 
term  forecasts  are  likely  to  have  tremen¬ 
dous  planning  value.  Consider  the  follow¬ 
ing  notional  example  illustrated  in  Figure 
2. 

System  XYZ  was  fielded  one  year  ago. 
The  original  release  had  some  problems 
with  one  of  its  interfaces,  and  as  a  result 


was  unable  to  pass  JITC  testing.  Never¬ 
theless,  System  XYZ  was  fielded  because 
it  provided  significant  capability  to  the 
Unified  Commands.  Unfortunately,  its 
failures  have  limited  its  operational  em¬ 
ployment.  Therefore,  its  current  interoper¬ 
ability  readiness  status  is  red  —  it  has  both 
operational  and  acquisition  problems. 

A  software  fix,  just  submitted  for  JITC 
testing,  has  undergone  favorable  initial 
review  and  is  expected  to  resolve  both  the 
operational  and  acquisition  problems. 
JITC  testing  will  be  completed  in  three 
months,  so  at  that  time  we  project  that  its 
interoperability  readiness  status  will  be 


status  projection 


Fielding 


Fix 

Tested 
<^>  Fielded 


Upgrade 

<^>  Due 
^  Expected 

<3>  Needed 


1 


Figure  2.  Sample  Timeline 
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orange.  Three  months  later  the  software 
fix  will  be  incorporated  in  all  fielded  units; 
so  in  six  months,  we  project  the  inter¬ 
operability  readiness  status  will  be  green. 

Meanwhile,  however,  System  XYZ  was 
fielded  with  a  preplanned  product  im¬ 
provement  strategy,  because  the  opera¬ 
tional  requirements  were  expected  to  in¬ 
crease  dramatically  over  the  course  of  the 
next  few  years.  The  upgraded  system  is 
due  in  two  years,  but  because  the  current 
system  implemented  a  few  features  early, 
it  will  actually  continue  to  function  effec¬ 
tively  for  the  next  30  months.  As  a  result 

of  a  funding 
cut,  the  PM  is 
already  warn¬ 
ing  that  the  up¬ 
grade  will  be 
three  months 
late.  Therefore, 
in  24  months 
we  expect  the 
system  to  enter  the  yellow  state,  where  it 
is  behind  the  acquisition  cycle  but  still 
meets  operational  requirements.  In  27 
months  when  the  next  version  is  released, 
it  will  become  green,  and  will  stay  green 
at  30  months  because  the  system  will  still 
meet  requirements. 

Applying  the  Basis 


The  section  above  describes  how  each 
system  can  be  graded  using  a  modified 
stoplight-style  scheme.  Systems  graded 
green  and  yellow  meet  their  operational 
requirements,  although  yellow  systems 
warn  of  possibly  overspecified  require¬ 
ments  or  potential  future  problems.  Sys¬ 
tems  graded  orange  and  red  limit  opera¬ 
tions.  Red  systems  indicate  problems  in 


requirements  implementation,  whereas 
orange  systems  indicate  a  (potentially 
harder  to  solve)  problem  in  requirements 
definition. 

The  problem  with  this  scheme  is  that  it 
may  not  provide  adequate  insight  into  the 
etiology  (root  cause)  of  the  problem.  Sys¬ 
tems  with  multiple  interfaces  may  be 
coded  red  because  of  problems  with  only 
one  interface.  Instead  of  grading  systems, 
it  may  be  necessary  to  unroll  this  rating 
and  grade  the  interfaces  between  systems. 
Each  pair  of  interacting  systems  could  be 
given  a  color  code  based  on  the  scheme 
described  above. 

The  advantage  to  grading  interfaces  is 
a  more  fine-grained  understanding  of  not 
only  the  systemic  cause  of  the  problem, 
but  also  of  its  operational  impact.  A  red 
system  may  perform  perfectly  well  in  a 
large  number  of  operational  scenarios,  if 
the  most  commonly  required  interfaces  are 
not  the  causes  of  the  problem.  Conversely, 
it  may  have  little  operational  value,  if  the 
most  useful  interfaces  are  the  ones  with 
the  problem. 

The  disadvantage  to  grading  interfaces 
is  the  dramatic  increase  in  the  magnitude 
of  the  problem  space.  With  the  increased 
interconnections  between  systems  in  the 
network-centric  environment  predicted  by 
JV2010,  we  can  expect  increasing  num¬ 
bers  of  interactions  between  system  pairs. 
Whereas  the  difficulty  of  measuring  readi¬ 
ness  for  systems  increases  with  the  num¬ 
ber  of  systems,  the  difficulty  of  measur¬ 
ing  readiness  for  interfaces  increases  with 
the  square  of  the  number  of  systems. 

One  way  to  mitigate  this  problem  would 
be  to  namow  the  number  of  interfaces  con¬ 
sidered.  Instead  of  grading  all  interacting 
system  pairs,  it  may  be  useful  to  begin  by 
evaluating  the  readiness  of  each  system  in 
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the  problem  space.  For  red  and  orange 
systems,  one  might  next  evaluate  each 
interaction  with  another  system  using  the 
readiness  reporting  model,  and  then 
develop  remediation  plans  on  a  per- 
interface  basis.  Once  this  process  was  es¬ 
tablished,  yellow  systems  could  be  simi¬ 
larly  analyzed  to  determine  whether  the 
requirements  were  overspecified  or  the 
systems  underutilized,  or  whether  (as  in 
our  earlier  example)  the  yellow  status  was 
a  transient  situation  that  was  not  cause  for 
great  alarm. 

Aggregate  Measures 


Clearly,  these  readiness  measures,  by 
themselves,  can  provide  a  useful  tool  in 
helping  to  address  the  DoD’s  interoper¬ 
ability  efforts.  They  focus  attention  first 
on  those  systems  that  do  not  meet  opera¬ 
tional  requirements,  putting  emphasis  on 
meeting  the  warrior’s  needs.  In  addition, 
they  help  to  identify  problems  in  the  re¬ 
quirements  definition  process.  By  separat¬ 
ing  problems  of  implementation  from 
problems  of  definition,  the  authors  believe 
the  process  will  highlight  the  successes  of 
the  acquisition  community.  More  impor¬ 
tant,  by  identifying  problems  of  definition, 
they  will  help  to  focus  efforts  in  this  area, 
hopefully  helping  to  prevent  these  prob¬ 
lems  from  recurring  in  future  systems. 
Given  the  sometimes  long  loop  between 
requirements  definition  and  operational 
employment,  it  is  key  that  these  lessons 
are  fed  back  into  the  requirements  gen¬ 
eration  system  as  quickly  as  possible  so 
that  they  are  not  compounded. 

However,  it  is  also  important  to  have  a 
general  measure  of  the  overall  health  of 
C4ISR  interoperability  in  the  DoD.  The 


readiness  reporting  measures  can  be  used 
to  provide  such  an  aggregate  measure. 

The  simplest  way  of  doing  this  is  to 
measure  the  percentage  of  systems  that  are 
classified  in  each  color.  Of  particular  in¬ 
terest  would  be  the  percentage  of  green 
and  yellow  systems  (the  system  opera¬ 
tional  interoperability  readiness  rate),  and 
the  percentage  of  green  and  orange  sys¬ 
tems  (the  system  acquisition  interoper¬ 
ability  success  rate).  It  might  also  be  in¬ 
teresting  to  consider  the  percentage  of 
green  and  red  systems,  which  indicates  the 
correlation  between  acquisition  success 
and  operational  readiness  —  perhaps  pro¬ 
viding  an  indicator  of  the  health  of  the 
acquisition  process  (including  the  require¬ 
ments  process)  as  a  whole.  See  Figure  3 
for  a  visual  depiction  of  these  rates. 

This  simple  method  of  aggregating  the 
scores  is  unambiguous.  But  does  it  pro¬ 
vide  a  fair  ranking  of  the  interoperability 
readiness  of  our  C4ISR  systems?  Because 
every  system  has  equal  weight,  a  small  sys¬ 
tem  fdling  a  relatively  small  niche  in  the 
combat  environment  (for  example,  a  medi¬ 
cal  field  supply 
reporting  sys¬ 
tem)  is  rated 
equally  with  a 
large  system 
that  interacts 
with  many  other 
battlefield  sys¬ 
tems  (for  ex¬ 
ample,  an  AWACS  aircraft).  This  may  not 
be  completely  appropriate. 

For  that  reason,  it  may  be  desirable  to 
assign  weights  to  each  of  the  systems  in 
our  problem  domain.  This  could  be  done 
using  any  number  of  methods,  taking  into 
account  factors  including,  but  not  limited 
to,  number  of  interfaces,  number  of 
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Figure  3.  Aggregate  Measures 
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deployed  units,  or  criticality  of  the  system 
as  rated  by  operational  commanders. 

The  percentages  described  above  could 
then  be  calculated  with  each  system  con¬ 
tributing  a  weighted  score.  (We  refer  to 
these  measures  as  weighted  system  opera¬ 
tional  interoperability  readiness  rates,  etc.) 
This  might  more  accurately  reflect  the 
state  of  the  department.  On  the  other  hand, 
it  might  tend  to  mask  problems  with  rela¬ 
tively  small,  yet  important  systems,  if  the 
weights  are  not  carefully  constructed. 

An  alternative  to  using  the  system  readi¬ 
ness  indicators  would  be  to  use  the  inter¬ 
face  indicators.  Again,  these  could  each 
be  given  equal  weight,  or  could  be 
weighted  based  on  the  importance  of  the 
particular  interaction  between  each  system 
pair.  We  refer  to  these  as  interface  inter¬ 
operability  readiness  rates  and  weighted 
interface  interoperability  readiness  rates, 
respectively. 

Clearly,  to  gain  value  from  these  aggre¬ 
gate  metrics,  they  must  be  maintained.  For 
them  to  be  maintained,  the  readiness 
scores  for  each  system  (or  interface)  must 
be  maintained.  It  is  also  important  to  real¬ 
ize  that  as  new  systems  are  fielded,  they 
become  legacy  systems  and  must  be  added 
to  the  scoring  system. 

Perhaps  the  best  way  to  roll  up  these 
figures  would  be  to  use  them  to  score  the 
ability  of  U.S.  forces  to  meet  various  mis¬ 
sion  scenarios.  By  developing  notional 
architectures  for  operational  employment 
of  our  forces,  the  interoperability  readi¬ 
ness  reporting  measures  could  be  used  in 
the  same  way  existing  readiness  measures 
are  used.  In  this  case,  typically,  the  lowest 
score  would  dominate  and  be  carried  up 
to  a  higher  level.  This  would  provide  valu¬ 
able  insight  into  the  ability  of  C4ISR  sys¬ 
tems  to  meet  national  strategic  needs,  and 


would  help  to  justify  funds  for  improve¬ 
ments  where  they  are  required. 

Discussion:  Solving  the  Problem 


With  this  collection  of  metrics  we  can 
now  propose  a  road  map  for  a  department 
wide  approach  to  addressing  the  C4ISR 
interoperability  problem.  First,  identify  an 
agency  with  responsibility  for  overseeing 
the  road  map.  Second,  under  the  direction 
of  the  lead  agency,  develop  an  authorita¬ 
tive  list  of  systems  that  fall  into  the  prob¬ 
lem  space  identified  in  this  paper,  together 
with  an  institutionalized  process  for  main¬ 
taining  that  list.  Third,  develop  readiness 
scores  (and,  possibly,  weights)  for  each 
system  on  the  list,  together  with  an  insti¬ 
tutionalized  process  for  maintaining  this 
data.  Fourth,  using  the  first  consolidated, 
across-the-board  data  set,  measure  the  cur¬ 
rent  state  of  our 
legacy  C4ISR 
interoperability 
problem.  Final¬ 
ly,  set  realistic 
goals  for  im¬ 
proving  this 
state,  allocating 
the  resources  re¬ 
quired  to  realize 
those  goals,  and  measuring  progress  along 
the  way.  This  section  will  discuss  this 
approach  in  more  detail. 

The  issues  surrounding  C4ISR 
interoperability  affect  the  services,  defense 
agencies,  and  the  unified  acquisition  com¬ 
munity.  In  addition,  success  in  this  effort 
ultimately  requires  the  ability  to  prioritize 
problems  across  the  military  community 
and  to  allocate  resources  in  support  of  the 
priorities.  The  authors  believe  that  this 
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function  is  fundamentally  related  to  Joint 
Forces  Command’s  UCP-99  responsibili¬ 
ties  as  joint  forces  trainer,  provider,  and 
integrator,  and  that  application  of  this 
metric  is  consistent  with  and  supportive 
of  their  other  objectives. 

As  its  first  task,  the  lead  office  would 
need  to  compile  a  master  list  of  mission 
essential  C4ISR  systems  with  interoper¬ 
ability  requirements.  An  approach  for 
doing  this  is  outlined  in  the  first  section 
of  the  paper.  Just  as  important  as  compil¬ 
ing  the  initial 
list,  the  organi¬ 
zation  must  put 
in  place  a  pro¬ 
cess  for  main¬ 
taining  the  list, 
involving  all  of 
the  various  en¬ 
tities  throughout  the  DoD  that  field  C4ISR 
systems.  This  encompasses  a  large  array 
of  organizations,  including  Communica¬ 
tions  and  Electronics  Command 
(CECOM),  Space  and  Warfare  Systems 
Command  (SPAWAR),  Electronic  Sys¬ 
tems  Command  (ESC),  Naval  Air  Systems 
Command  (NAVAIR),  Naval  Sea  Systems 
Command  (NAVSEA),  and  PEOs  within 
the  Air  Force  and  Army,  DISA,  Defense 
Intelligence  Agency  (DIA),  National  Se¬ 
curity  Agency  (NS  A),  Special  Operations 
Command  (SOCOM),  and  others.  This 
process  would  most  likely  involve  integra¬ 
tion  of  existing  processes,  rather  than 
development  of  new  ones;  nevertheless, 
given  the  number  of  developing  organiza¬ 
tions,  this  represents  a  potentially  non¬ 
trivial  effort.  Even  by  itself,  however,  this 
task  will  likely  result  in  a  valuable  resource 
by  identifying  all  systems  in  this  class, 
together  with  the  responsible  agency. 


With  the  master  list  in  hand,  this  office 
could  then  begin  to  assess  the  readiness 
of  each  of  these  systems.  This  rating  need 
not  be  done  by  a  single  office,  but  could 
be  delegated  to  subordinate  agencies.  The 
key  to  this  evaluation,  in  fact,  is  to  ensure 
that  the  operational  community  evaluates 
the  operational  requirements,  while  the 
acquisition  community  evaluates  the  ac¬ 
quisition  requirements.  The  office,  of 
course,  would  be  responsible  for  provid¬ 
ing  sufficient  oversight  to  ensure  honest 
responses  were  provided.  In  addition  to 
understanding  “today’s”  situation,  data 
should  be  collected  on  the  projected  status 
over  the  medium  term,  perhaps  3-5  years. 
As  described  in  the  example  earlier, 
changes  in  status  can  sometimes  be  pre¬ 
dicted  in  advance,  and  knowing  when 
these  changes  will  take  place  is  important 
data  for  decision  makers. 

Once  again,  determining  the  status  of 
these  systems  is  not  enough.  The  office 
must  put  in  place  procedures  for  maintain¬ 
ing  this  information.  The  challenge  here 
comes  from  the  large  number  of  players. 
Although  much  of  this  information  exists 
within  the  department,  making  it  acces¬ 
sible  in  normalized  form  in  a  centralized 
location  would  represent  a  key  contribu¬ 
tion  to  dealing  with  the  interoperability 
problem. 

Once  the  list  of  systems  and  the  data 
on  each  system  is  available,  and  an  insti¬ 
tutional  process  for  maintaining  it  is  in 
place,  the  office’s  work  becomes  at  once 
simpler  and  more  significant.  In  addition 
to  keeping  the  process  running  (which  is 
likely  to  remain  nontrivial  in  light  of  the 
propensity  for  reorganization  within  the 
federal  government),  the  office  must  com¬ 
pile  aggregate  statistics  (including  projec¬ 
tions  of  the  aggregate  statistics).  More 
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important,  the  office  must  drill  down  into 
the  interfaces  of  problem  systems,  determin¬ 
ing  the  true  sources  of  problems,  and 
engage  the  necessary  players  to  effect 
solutions.  This  could  be  done  both  to  solve 
present  problems  and  to  prevent  or 
mitigate  problems  that  may  be  anticipated. 

Of  course,  in  addition  to  the  depart¬ 
ment-wide  statistics,  the  data  could  be  cut 
along  different  lines.  One  could  consider 
the  interoperability  readiness  of  all  sen¬ 
sors,  all  shooters,  all  systems  within  a 
particular  area  of  interest,  or  all  systems 
produced  by  a  particular  agency.  Using 
interface  data,  one  could  even  construct  a 
picture  of  the  “interoperability”  between 
two  agencies  by  considering  the  status  of 
interfaces  for  systems  produced  by  the  two 
agencies.  This  might  identify  instances  in 
which  institutional  obstacles  played  a  role 
in  the  observed  interoperability  problems. 
Equally  important,  it  would  likely  high¬ 
light  the  tremendous  levels  of  cooperation 
between  many  of  the  agencies  acquiring 
C4ISR  systems. 

Armed  with  hard  data,  “ground  truth” 
can  be  provided  to  the  interoperability 
debate.  Realistic  goals  could  be  set,  and 
the  resources  to  achieve  these  goals  could 
be  allocated  to  the  organizations  in  the  best 
positions  to  do  so.  Most  important,  pro¬ 
gress  toward  these  goals  could  be  objec¬ 
tively  measured.  The  process  would  sepa¬ 
rate  requirements  issues  from  acquisition 
issues,  offering  opportunities  for  improv¬ 
ing  both  of  these  systems  when  systemic 
factors  are  found  to  have  contributed  to 
problems.  At  the  same  time,  it  would 
ideally  protect  all  parties  from  recrimina¬ 
tion  by  focusing  the  entire  community  on 
solving  the  problem  —  delivering  C4ISR 
systems  that  meet  the  interoperability 
demands  of  our  warriors. 


Conclusion 


One  can  easily  argue  that  America’s 
unrivalled  dominance  on  the  battlefields 
of  the  late  20th  century  is  due  largely  to 
the  success  of  our  acquisition  system,  even 
in  light  of  the  declining  defense  budgets 
of  the  last  decade. 

Our  ability  to  attain  full-spectrum  domi¬ 
nance  in  the  2 1  st  century  will  rely  heavily 
on  our  C4ISR  infrastructure  (Chairman  of 
the  Joint  Chiefs  of  Staff,  2000).  Interoper¬ 
ability  of  our  C4ISR  systems  is  essential 
to  achieving  this  goal.  We  are  doing  well 
in  this  area,  but  we  can  do  better. 

That  which  is  measured  improves.  We 
will  never  fully  eliminate  interoperability 
problems,  because  evolving  operational  re¬ 
quirements  will  continue  to  challenge  the 
developers  of  C4ISR  systems.  By  its  very 
nature,  our  enterprise  will  always  be  push¬ 
ing  the  limits  of  technology,  generating 
new  problems  even  as  the  old  ones  are 
solved.  It  is  important  to  show  progress, 
analyze  the  drivers  behind  our  interoper¬ 
ability  problems,  and  apply  maximum 
effort  at  the  point  of  greatest  leverage  to 
solve  them.  Only  in  this  way  can  we  pro¬ 
vide  the  greatest  possible  utility  to  the 
soldier  on  the  battlefield. 

We  have  developed  a  simple,  readiness¬ 
reporting  style  method  of  measuring 
interoperability.  We  also  have  proposed  a 
method  for  aggregating  the  data  in  a 
manner  that  will  facilitate  tracking  pro¬ 
gress  on  a  DoD-wide  basis.  Finally,  we 
have  outlined  a  mechanism  for  applying 
the  metrics  within  the  DoD  to  facilitate 
solution  of  the  problem.  It  is  our  sincere 
hope  that  this  road  map  will  generate 
open  discussion  within  the  department  that 
will  ultimately  lead  to  a  more  rigorous 
approach  to  interoperability. 
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